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Introduction 

In March 2005, the College Board introduced a revised SAT® 
that included a writing section and discontinued use of the 
SAT Subject Test in Writing. To date, well over two million 
students have taken the new SAT, and this paper examines 
research and statistical results from the new writing section. 

The writing section is composed of multiple- choice 
questions and an essay. Each form of the SAT includes a 
25-minute and a 10-minute writing multiple-choice section, 
and a 25-minute essay. The 49 multiple- choice items are com- 
bined to produce a scaled writing subscore from 20 to 80. The 
essay is scored by two trained readers on a 1-6 scale; those 
scores are combined to produce a raw subscore of 2-12 (stu- 
dents can receive a score of zero under special circumstances 
described below). If two readers disagree by more than 1 point, 
the essay is sent to an expert reader to determine the final score 
on the 1-6 scale and this is doubled to produce a 2-12 raw sub- 
score. The multiple-choice and essay subscores are combined to 
form a total score for the writing section on the 200-800 scale. 

Essay Scores 

We examined all essay scores' (1,376,745) taken by students in 
the 2006 College-Bound Seniors cohort and found: 

1. Fewer than 2,000 students (one-tenth of a percent) 
received a score of zero on the essay. A score of zero is 
given when students write in a different language, are off 
topic, only repeat the prompt, or leave the essay blank. 


2. The mean score was 7.2 with a standard deviation of 1.7 
(males, 7.1; females, 7.4), and the median score was 7. 

3. Sixty-nine percent of essays received a score between 6 
and 8; 80 percent of the essays received a score between 
6 and 9 (see Figure 1). The distribution of essay scores 
on the SAT and other tests is different from typical score 
(or normal) distributions in that, typically, relatively few 
students score at the extreme ends of the scale. This is 
because only one prompt is used, and readers give few 
scores of 1 or 6. 

4. The inter-rater reliability of the scores from two raters 
on the same essay was .77 to .81 for the 2006 College- 
Bound Seniors cohort. During the test administrations 
from October 2005 to March 2006, the average number 
of essays requiring a third reader ranged from approxi- 
mately 2.8 percent to 4.1 percent. The alternate forms 



Figure 1. 2006 College-Bound Seniors SAT essay score 
distribution. 


^Essay scores are reported for all essays completed by College-Bound Seniors in 2006, not the highest essay score or total writing score, which is how other 
College-Bound Senior data have been reported. (For example, if a student took the SAT three times, all three scores were used in the above calculations, not just 
the highest score.) Also note that some students in this cohort took the SAT without writing prior to March 2005. 
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Table 1 


Overall Reliability of Scores for the Three SAT Sections 


SAT Scores 

Reliability 

Critical Reading 

.91 to .93 

Mathematics 

.92 to .93 

Writing 

.86 to .92 


reliability of the essay (correlation between essays for 
an individual student) was .67. However, the overall 
reliability^ of the scores for the three SAT sections are 
significantly higher and justify their use for high -stakes 
admissions decisions (see Table 1). 

5. The distribution of scores by ethnicity reveals smaller sub- 
group differences than is typically found with other cog- 
nitive tests. That is, an average difference of 1.0 point on 
the essay between the highest and lowest scoring groups 
translates to a standardized difference of .59. Score gaps on 
the essay are smaller than score gaps on the critical read- 
ing, mathematics, or writing multiple -choice sections due 
to a variety of factors, such as students having to complete 
only one essay compared to having to complete numerous 
items on the other sections; relatively few Is and 6s given; 
reduced scale range; etc. Standardized differences^ on the 
essay were also computed and are reported below (see 
Figure 2 and Table 2). 

Table 2 


Standardized Differences on SAT Essay Scores by 
Subgroups 


Subgroup 

Mean 

SD 

Essay Standardized 
Difference 

Female 

7.4 

1.6 

0.18 

Male 

7.1 

1.7 

-0.18 

American Indian 

7.0 

1.6 

-0.12 

Asian American 

7.5 

1.8 

0.18 

African American 

6.5 

1.6 

-0.41 

Hispanic 

6.8 

1.6 

-0.24 

White 

7.5 

1.6 

0.18 

English 

7.3 

1.6 

0.06 

English and Another 
Language 

7.2 

1.7 

0.00 

ESL 

6.9 

1.8 

-0.18 



Subgroup 


Figure 2. Mean SAT essay scores by subgroups. 

Relationship Between Essay 
Features and SAT Writing 
Scores 

There have been concerns that essay features that should not 
be related to essay scores, such as length, number of para- 
graphs, number of words, and quality of handwriting, result in 
artificially inflated scores. Even critics of the SAT writing sec- 
tion have acknowledged that some features like the length of 
an essay and number of words in an essay will correlate with 
an essay score, but they have insisted that the correlation on 
the SAT essay was significantly above what would be expect- 
ed. Their claims were based on a small number of essays that 
were pulled from training papers to illustrate essays at each 
scale point and not a random sample of papers. 

In 2006, we conducted a study of 6,498 essays from 14 
prompts (East and West Coast) across multiple administrations. 
Temporary staff were hired and trained to count the number of 
words and paragraphs as well as to code essays on other such 
surface features. Researchers examined the relationship between 
scores and each feature. Table 3 illustrates the correlations 
between essay scores and SAT writing composite scores. 

Table 3 


Correlation Between Essay Features and SAT Writing 
Scores 


Essay Feature 

SAT Essay Score 

SAT Writing Score 

Number of Words 

0.63 

0.43 

Number of Paragraphs 

0.38 

0.26 


^ Reliabilities for the mathematics, critical reading, and multiple-choice writing sections were calculated using the KR-20 formula. The overall reliability of the 
writing section was computed by taking a weighted average of the two subsections’ reliability estimates. 

^ For females, standardized difference is calculated as (Female Mean minus Male Mean)/Total Standard Deviation. For males, the same formula applies except 
the female mean is subtracted from the male mean. For all other subgroups, standardized difference is calculated as (Subgroup Mean minus Total Mean)/Total 
Standard Deviation. 
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These correlations are in line with those from other 
studies of essay scoring. We also found no practical difference 
among the mean score for papers written in the first person 
(6.9) compared to papers not written in the first person (7.2); 
cursive (7.2) versus printed (7.0); and papers using academic 
examples (7.4) versus personal examples (7.0). About 8 per- 
cent of papers used an obvious shell (e.g., five -paragraph 
essay). Additional research examining other surface and con- 
tent features is under way (Kobrin et al., in preparation). 

Writing Section Score and 
Subgroup Differences 

The mean writing section score was 497 for 2006 College- 
Bound Seniors compared to 503 for critical reading, and 518 
for mathematics. The gender difference on the writing section 
was 11 points in favor of females (491 for males versus 502 
for females), in contrast to mathematics and critical reading 
scores, where males scored higher than females. Performance 
on the SAT writing section by racial/ethnic group and first- 
language group are displayed in Figures 3 and 4. 

Table 4 displays the score gap (or raw score difference), 
as well as the standardized difference, on the writing section 
by gender, race/ethnicity, and first-language group. The score 
gap was similar or smaller than those on the critical reading 
section for African American, ESL (first language other than 
English), and Mexican American students.'* This illustrates 
that the addition of the writing section did not increase 
disparities or score differences for most racial/ethnic and 
language groups. American Indian, Puerto Rican, and other 



Race/Ethnicity 


Figure 3. Mean performance on the SAT writing section 
by racial/ethnic group. 


Table 4 


Score Differences Across Gender, Race/Ethnicity, 
and First-Language Group 


Subgroup 

CR Raw 
Dijference 

W Raw 
Difference 

CR 

Standardized 

Difference 

w 

Standardized 

Difference 

Gender 





Females 

-3 

11 

-.03 

.10 

Males 

3 

-11 

.03 

-.10 

Race/Ethnicity 





American Indian 

-16 

-23 

-.14 

-.21 

Asian American 

7 

15 

.06 

.14 

African American 

-69 

-69 

-.61 

-.63 

Mexican American 

-49 

-45 

-.43 

-.41 

Puerto Rican 

-44 

-49 

-39 

-.45 

Other Hispanic 

-45 

-47 

-.40 

-.43 

White 

24 

22 

.21 

.20 

First Language 





English Speakers 

12 

9 

.11 

.08 

English and 
Another Language 

-18 

-7 

-.16 

-.13 

Another Language 

-36 

-28 

-.32 

-.26 


Hispanic groups had slightly larger differences on the writing 
section than on the critical reading section. 

Clearly, this is only the first year of data and additional 
data points are needed before drawing firm conclusions about 
the impact of writing on subgroup differences. Yet, based 
on more than 1.3 million students who took the new SAT, 
evidence supports the claims that including the SAT writ- 
ing score in admissions does not have a negative impact or 
increase the disparities among majority and minority groups. 



Speakers Another Language Language 

First-Language Group 

Figure 4. Mean performance on the SAT writing section 
by first-language group. 


‘ A reduction in the raw difference does not guarantee a reduction in the standardized difference due to different standard deviations across test sections. 
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Validity of Writing 


Based on the data from the 2006 College-Bound Seniors cohort, 
we examined the relationship between the new writing section 
and the other sections of the SAT. As expected, writing and 
critical reading were more highly correlated (.85) than math- 
ematics and critical reading (.71) and mathematics and writing 
(.72). For the writing subscores, the multiple -choice section 
and essay correlated .55. Critical reading correlated .55 with the 
essay and .84 with the writing multiple- choice section. 

As for the predictive validity of the SAT writing section, 
the College Board funded two projects examining the predictive 
validity of SAT writing; the results are summarized below. The 
first study conducted by Kobrin, Camara, and Milewski (2002) 
found that the SAT Subject Test in Writing had greater predic- 
tive validity than the SAT Reasoning Tesf” for Asian American 
and Hispanic students. For students at four campuses of the 
University of California, the SAT Subject Test in Writing had a 
higher validity coefficient than the SAT Reasoning Test for all 
racial/ethnic groups, and was the single best predictor of first- 
year college grade point average (FGPA) for all ethnic groups 
except American Indian and African American students. 

The second study, by Norris, Oppler, Kuang, Day, and 
Adams (2006), also provides evidence for the validity of the 
SAT writing section. A pilot version of the SAT writing sec- 
tion was administered to 1,572 incoming freshmen at 13 col- 
leges and universities. Their scores on the verbal and math- 
ematics sections of the SAT were also obtained in order to 
assess the incremental validity of the SAT writing section for 
the prediction of FGPA and English composition grade point 
average (ECGPA). The results revealed that the SAT writing 
section correlated 0.46 with first-year college GPA and 0.32 
with English composition course grades, after correcting for 
range restriction (see Table 5 below, which is taken directly 



SAT Writing Score 

□ FGPA 2.5 I FGPA 3.0 □ FGPA 3.5 

Figure 5. Percentage of students in SAT Writing Pilot 
Validity Study earning first-year college GPAs at different 
levels by SAT writing score holding constant high school 
GPA (>3.7) and SAT verbal + math score (1210-1400). 

from Norris et al., 2006). After controlling for high school 
grade point average (HSGPA) and SAT mathematics and ver- 
bal scores, the incremental validity of the SAT writing section 
for predicting FGPA was .01. 

Reanalyzing the same data set from Norris, Oppler, 
Kuang, Day, and Adams (2006), Figure 5 shows the percent- 
age of students participating in the SAT Writing Pilot Validity 
Study with high school GPAs > 3.7 and SAT verbal -f math 
scores between 1210 and 1400, who earned a first-year college 
GPA at different levels according to their SAT writing compos- 
ite score on the prototype writing test with essay. It is clearly 
shown that holding constant high school grades and SAT verbal 
and mathematics scores, SAT writing scores make a difference 
in the percentage of students earning higher college grades. 

In sum, these results provide the first evidence for the pre- 
dictive validity of the SAT writing section. The GoUege Board is 
recruiting up to 100 institutions this year to participate in a valid- 


Table 5 


Weighted-Average Correlations for AH Predictors with First-Year College GPA and English Composition GPA 


Predictor 

FGPA 

ECGPA 

N 

Corrected 

Uncorrected 

N 

Corrected 

Uncorrected 

SAT-V 

1,248 

0.49 

0.32 

891 

0.30 

0.20 

SAT-M 

1,248 

0.47 

0.29 

891 

0.23 

0.10 

SAT-T 

1,248 

0.51 

0.35 

891 

0.28 

0.17 

SAT-ES 

1,248 

0.20 

0.16 

891 

0.18 

0.14 

SAT-MC 

1,248 

0.45 

0.30 

891 

0.31 

0.22 

SAT-W 

1,248 

0.46 

0.32 

891 

0.32 

0.24 

HSGPA 

1,248 

0.43 

0.38 

891 

0.35 

0.32 


Note: SAT-V=SAT verbal score, SAT-M=SAT mathematics score, SAT-T=SAT total score, SAT-ES=SAT essay score, SAT-MC=SAT writing multiple- 
choice score, SAT-W=SAT writing score, and HSGPA=cumulative high school grade point average. 
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ity study for the entering class of 2006, which is the first cohort 
to complete the new writing section. We will examine both the 
predictive and placement validity of the test scores. 

Consequential Validity 

The SAT writing section was added with the intent that it 
would reinforce the importance of writing skills throughout 
a student’s education, and that it would support the academic 
achievement of all students, bolstering their chances for aca- 
demic success in college. 

In collaboration with research consultant Dr. Richard 
Noeth, the Research and Analysis department completed a study 
to determine the near-term impact (or consequential validity) of 
the SAT writing section on K-12 education. The overall purpose 
of this study was twofold: (1) to learn about changes in writing 
instruction across the nations K-12 education system, including 
changes in programs, strategies, and support over the past three 
years; and (2) to describe the near-term impact on K-12 educa- 
tion of the addition of the writing section to the SAT. 

To accomplish these goals, large national samples 
(selected across all College Board regions) of both English/ 
language arts high school teachers and school district cur- 
riculum directors were surveyed online in November 2006. 
Letters inviting participation in the surveys were e-mailed 
to a total of 10,918 curriculum directors (CDs) and 56,384 
English/language arts (ELA) teachers. A total of 764 (7 
percent) of curriculum directors and 4,888 (8.7 percent) of 
teachers completed the survey. Survey content focused on 
changes in writing attitudes and expectations, the teaching of 
writing, student learning related to writing, and changes in 
K-12 resources dedicated to writing. Preliminary results do 
provide support for assertions that adding writing to the SAT 
has increased both instructional time devoted to writing and 
the number of writing assignments in high school English/ 
composition courses. 

The vast majority of the survey respondents (88 per- 
cent of teachers and 93 percent of curriculum directors) 
reported that writing has become more of a priority in their 
school/district over the past three years. Respondents were 
asked to indicate the importance of writing to their curricu- 
lum three years ago, and today. Both ELA teachers and CDs 
reported that writing has become a more important part of 
the curriculum. The percentage indicating that writing was a 
very important or the most prominent part of their curricu- 
lum increased from 30-37 percent three years ago to 62-70 
percent today. 


Table 6 shows some selected preliminary results of the 
survey. Survey respondents were asked to indicate the extent 
to which the SAT writing section has had an impact overall, 
and on teacher, administrator, student, and school board 
attitudes; curriculum and teaching methods; student learning 
and performance; and resources allocated to writing. More 
than half of the responding ELA teachers and CDs indicated 
that the SAT has had at least a minor impact on all areas except 
for resources allocated to writing. ELA teachers were gener- 
ally more inclined than CDs to indicate that the SAT writing 
section has had an impact. A full report of survey findings 
will be available by spring 2007 at www.collegeboard.com/ 
research/home/. 

Writing May Help Colleges 
Make Better A(imissions 
Decisions; Students 
with Discrepant Scores 
on Critical Reading and 
Writing 

The value of adding a writing section to the SAT has been 
raised by some, given that most students perform similarly on 
critical reading and writing tests. In fact, for the 2006 College- 
Bound Seniors cohort, the correlation between critical read- 
ing and writing was .85. However, a high correlation does not 
necessitate that students who perform high or low on one 
test must perform similarly on the other test. Using the 2006 
College-Bound Seniors cohort, we looked at the number of 
students who scored differently in critical reading and writing 
and their demographic breakdown. 

In order to determine whether a student scored discrep- 
antly on the two sections, we first standardized the scores so 
that they were on the same scale. Then, we classified students 
as discrepant if their critical reading and writing scores differed 
by one standard unit or more. Below is a summary of the new 
variables we created for the analyses. As displayed in Table 7, 
the difference between critical reading and writing scores (CR- 
Writing) had a mean of 0.00 and a standard deviation of .56, 
indicating that most students do perform about the same on 
both. However, at least one student scored 5.02 standard units 
higher on writing than on critical reading and another scored 
4.74 standard units higher on critical reading than on writing, 
indicating that discrepant scores definitely occur. 
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Table 6 


Preliminary Results of a Study on the Near Impact of the SAT Writing Section on K-12 Education 


Survey Item 

ELA % 

CD% 

Teachers’ attitudes toward the importance of writing have become more positive 

76 

84 

Administrators’ attitudes toward the importance of writing have become more positive 

78 

90 

Students’ awareness of the importance of writing has become more positive 

80 

N/A 

School boards’ attitudes toward the importance of writing have become more positive 

N/A 

60 

Teachers have higher expectations with regard to writing 

85 

91 

Teacher-parent communications have included more discussion about writing 

49 

N/A 

At least one new writing course has been added in their school/district 

28 

31 

At least some additional class time has been spent on writing 

80 

81 

More class time has been spent on writing projects, assignments, analysis, and/or writing-related activities 

50 

69 

ELA teachers have given at least a few more in-class writing assignments 

86 

85 

Teachers other than ELA teachers have given more in-class writing assignments 

43 

62 

ELA teachers have given at least a few more outside-of-class writing assignments 

68 

67 

Teachers other than ELA teachers have given more outside-of-class writing assignments 

27 

42 

Writing has become more of a focus of the ELA curriculum 

76 

84 

Teaching methods have included more of a focus on writing 

82 

N/A 

Curricula in other subjects have included more of a focus on writing 

37 

60 

Teaching methods in other subjects have included more of a focus on writing 

32 

N/A 

Overall curriculum rigor has been increased by greater attention to writing 

74 

83 

Writing has become more closely tied to reading 

78 

84 

There has been more interdisciplinary collaboration 

44 

67 

More essay tests are given across the curriculum 

36 

48 

SAT-type essays are used 

60 

N/A 

There has been overall improvement in students’ writing skills 

75 

84 

There has been improvement in the writing skills of underserved, ESL, and at-risk students 

46 

62 

General writing programs have been expanded and/or implemented 

29 

41 

Special writing programs for underserved, ESL, or at-risk students have been expanded and/or implemented 

39 

44 

More resources have been allocated to writing 

34 

57 

There has been more professional development in writing 

64 

85 

Teachers have been given more dedicated time to grade writing assignments 

7 

14 

There have been expanded and/or new writing proficiency requirements in the district 

N/A 

59 


In order to determine how many students had discrep- 
ant scores, we took the absolute difference between ones criti- 
cal reading score and writing score |CR- Writing! and found 
that 99,692 students had a discrepant score of 1 standard unit 
or higher. In order to determine how many students were 

Table 7 


Comparison of Critical Reading and Writing Scores to 
Determine Discrepancies 


Test 

Minimum 

Maximum 

Mean 

SD 

Critical Reading (Standardized) 

-2.70 

2.63 

0.00 

1.00 

Writing (Standardized) 

-2.73 

2.79 

0.00 

1.00 

CR-Writing 

-5.02 

4.74 

0.00 

0.56 

|CR-Writing| 

0.00 

5.02 

0.44 

0.34 


better at writing and how many were better at critical read- 
ing, we took the difference between each students critical 
reading score as compared to his or her writing score in stan- 
dard units. A negative value indicates that the student scored 
higher on writing than on critical reading, whereas a positive 
value indicates that the student scored higher on critical read- 
ing than on writing. We found that 49,356 students scored 
1 or more standard units higher on writing than on critical 
reading and 50,336 students scored 1 or more standard units 
higher on critical reading than on writing. 

Next, we looked to see whether individuals with dis- 
crepant scores differed in terms of HSGPA, gender, and race/ 
ethnicity. Table 8 shows that the mean HSGPA was highest 
for the “Better at Writing” group and lowest for the “Better 
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Table 8 


Table 9 


Mean HSGPA by Score Discrepancy Category 


Score Discrepancy 

Mean 

HSGPA 

SD 

F 

p-value 

Better at Writing 

3.38 

0.63 

786.5 

<0.001 

Same on Both 

3.33 

0.63 



Better at Critical Reading 

3.22 

0.66 




at Critical Reading” group. A one-way analysis of variance 
(ANOVA) with planned contrasts revealed significant differ- 
ences among all three groups. 

As for gender, Figure 6 shows that almost twice as many 
females (31,005) were categorized as “Better at Writing” than 
were males (18,351). Similarly, almost twice as many males 
(31,694) were categorized as “Better at Critical Reading” than 
were females (18,642). This is not surprising given that females 
scored 11 points higher than males on writing and males 
scored 3 points higher than females on critical reading. 

In terms of ethnicity. Table 9 and Figure 7 demon- 
strate that the percentage of white students and American 
Indian/ Alaska Native students is higher in the “Better at 
Critical Reading” category, whereas the percentage of Asian, 
Asian American, or Pacific Islander students is higher in the 
“Better at Writing” category. All other racial/ethnic groups are 
roughly the same across categories. 

Coachability of the Essay 

When it was decided to add a writing component to the 
new SAT, many concerns were raised about the coachability 
of essays. In order to ensure the integrity of the test, it was 


Racial/Ethnic Makeup of Score Discrepancy Categories 


Ethnicity 

Better at Writing 

Same on Both 

Better at 
Critical Reading 

N 

% 

N 

% 

N 

% 

No response 

5,336 

10.81 

112,047 

8.78 

4,941 

9.82 

American Indian 
or Alaska Native 

241 

0.49 

8,075 

0.63 

437 

0.87 

Asian, Asian 
American, or 
Pacific Islander 

6,840 

13.86 

117,081 

9.17 

3,833 

7.61 

African American 
or Black 

4,960 

10.05 

133,100 

10.43 

5,238 

10.41 

Hispanic 

5,038 

10.21 

135,556 

10.62 

5,114 

10.16 

white 

24,841 

50.33 

724,807 

56.77 

29,010 

57.63 

Other 

2,100 

4.25 

46,024 

3.60 

1,763 

3.50 

Total 

49,356 

100 

1,276,690 

100 

50,336 

100 


imperative that the College Board assess the impact of coach- 
ing on test scores because of its possible detrimental effect on 
test validity. An external study conducted by Hardison and 
Sacked (2006) examining the coachability of essays on the 
SAT addresses this issue and is summarized below, followed 
by two proposals for future research on the topic. 

Hardison and Sackett’s (2006) study examined the 
effect of short-term coaching, which is the typical format of 
test-preparation companies, on essay performance, as well as 
its generalizability to other college writing assessments. They 
were interested in answering the question, “Does coaching 
just increase scores on the specific essay, or does coaching 
increase actual writing ability?” If coaching truly improves 
one’s writing ability rather than artificially inflating one’s 
score, then the validity of the test is preserved. 


70- 


60- 


50- 


0) 

S) 

iS 40- 
e: 

0 ) 

“ 30- 

<0 

o. 


20 - 


10 - 


I I Male O Female 


Better at Writing 


Same on Both 


Score Discrepancy 


Better at Criticai 
Reading 


1 Better at Writing Same on Both Better at Critical Reading | 















n tnc 

rm-i 


No American Asian, African Hispanic White Other 

Response Indian or Asian American 

Alaska American, or Black 
Native or Pacific 
Isiander 


Race/Ethnicity 


Figure 6. Gender makeup of score discrepancy 
categories. 


Figure 7. Racial/ethnic makeup of score discrepancy 
categories. 
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In the first part of the study, subject-matter experts 
(SMEs) developed a nine-hour coaching program in which 
they examined previous essays and determined what features 
of the essays resulted in higher scores. With that information, 
they developed strategies and rules for constructing a high- 
scoring essay. Finally, they compared their coaching program 
to that of professional test-preparation companies and found 
that there was substantial overlap in material, thus giving cre- 
dence to their newly developed program. 

In the second part of their study, they recruited 99 
University of Minnesota freshmen (77 percent female, 77 
percent white) and randomly assigned them to either the 
coaching group (treatment) or the control group. All par- 
ticipants completed two essay prompts, which were taken 
from the College-Level Examination Program* (CLEP*), 
as a pretest measure of writing ability, and provided self- 
reported SAT/ACT scores. The control group participants 
came back the next day and completed two more CLEP essay 
prompts and two other writing tasks. The treatment group 
attended the coaching program over the next two days and 
on the fourth day, completed two more CLEP essay prompts 
and two other writing tasks. All CLEP essays were scored by 
trained CLEP graders, and the additional writing tasks were 
scored by the SMEs. 

After controlling for pretest performance and admis- 
sions test scores, the treatment variable (training versus 
control) was a significant predictor of posttest scores and gen- 
eralizability scores. That is, training did significantly improve 
scores on both outcomes. It should be noted that the effect 
was not very large, with pretest scores being a much stronger 
predictor of posttest performance than training. Furthermore, 
the training group’s mean scores increased by only 0.1 1 on a 6- 
point scale, whereas the control group’s mean scores decreased 
by 0.11 on a 6-point scale from pretest to posttest. There was 
also a significant interaction between group membership and 
pretest scores on posttest performance, with training typically 
helping those who had low pretest scores and hindering those 
with high pretest scores. In sum, coaching did increase per- 
formance in the posttest prompts and in other writing tasks. 
These results suggest that writing samples on the SAT are 
susceptible to coaching, but score increases may reflect at least 
some improvement in overall writing ability. 

Two additional studies have been proposed to further 
investigate the effects of coaching on test scores. The purpose of 
the first study is to determine the impact on SAT scores when 
students have participated in The Official SAT Online Course™, 
which is offered through the SAT Readiness Program™. That is, 
we want to answer the question, “What is the average SAT score 
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change for students at schools participating in The Official SAT 
Online Course compared to a matched control group?” The 
purpose of the second study is to determine the use of a variety 
of programs, resources, and approaches designed to prepare 
students for the SAT, and to examine the relationship of these 
preparation activities to score changes. The research questions 
for this study are as follows: 

1. To what extent do various test-preparation approaches 
affect SAT scores? 

2. Are there features of the test that are sensitive to different 
approaches? 

3. How do products/services from different providers in a 
single category affect scores? 

4. How do products/services from different providers in 
different categories affect scores? 

In order to test these research questions, we will survey a 
nationally representative sample of SAT takers (juniors and 
seniors) to determine how students are preparing for the SAT, 
the amount of time they estimate partaking in each prepa- 
ration activity, and the provider of the type of preparation. 
Then, we will compare score changes on the different sections 
and features of the SAT by type of preparation, as well as com- 
pare the effects of products/ services from a single preparation 
category from different providers on scores and the effects of 
products/ services from different providers in different prepa- 
ration categories on scores. 

Discussion 

In preparation for the launch of the new SAT, the College 
Board, in addition to external researchers, has conducted 
and continues to conduct studies assessing the psychometric 
quality and impact of the new writing section. The focus 
of this research note is to summarize the research that has 
been conducted thus far on the new writing section. The 
evidence provided herein reveals that the new writing section 
has satisfactory psychometric quality in that its reliability is 
acceptable; it is significantly related to first-year college GPA 
and college English grades; it has been perceived to impact 
high school curriculum; and it does not result in larger score 
disparities among racial/ethnic groups. However, much of the 
research described above is based on individual studies and 
may not generalize to the total test-taking population, and 
there are still numerous research questions left unanswered. 
Furthermore, since this is the first year in which the new SAT 
was administered, we are limited in what we can say about 
its predictive validity since the students are just now entering 


college. As data on the SAT Reasoning Test continue to accu- 
mulate, we will continue to monitor and assess the psycho- 
metric quality of the new SAT writing section and its impact 
on educational practices and policies. 
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